Entropy Measures vs. Kolmogorov Complexity
نویسندگان
چکیده
منابع مشابه
Entropy Measures vs. Kolmogorov Complexity
Kolmogorov complexity and Shannon entropy are conceptually different measures. However, for any recursive probability distribution, the expected value of Kolmogorov complexity equals its Shannon entropy, up to a constant. We study if a similar relationship holds for Rényi and Tsallis entropies of order α, showing that it only holds for α = 1. Regarding a time-bounded analogue relationship, we s...
متن کاملShannon Entropy vs. Kolmogorov Complexity
Most assertions involving Shannon entropy have their Kolmogorov complexity counterparts. A general theorem of Romashchenko [4] states that every information inequality that is valid in Shannon’s theory is also valid in Kolmogorov’s theory, and vice verse. In this paper we prove that this is no longer true for ∀∃-assertions, exhibiting the first example where the formal analogy between Shannon e...
متن کاملInequalities for Shannon Entropy and Kolmogorov Complexity
It was mentioned by Kolmogorov (1968, IEEE Trans. Inform. Theory 14, 662 664) that the properties of algorithmic complexity and Shannon entropy are similar. We investigate one aspect of this similarity. Namely, we are interested in linear inequalities that are valid for Shannon entropy and for Kolmogorov complexity. It turns out that (1) all linear inequalities that are valid for Kolmogorov com...
متن کاملObservational Modeling of the Kolmogorov-Sinai Entropy
In this paper, Kolmogorov-Sinai entropy is studied using mathematical modeling of an observer $ Theta $. The relative entropy of a sub-$ sigma_Theta $-algebra having finite atoms is defined and then the ergodic properties of relative semi-dynamical systems are investigated. Also, a relative version of Kolmogorov-Sinai theorem is given. Finally, it is proved that the relative entropy of a...
متن کاملSharp Asymptotics of the Kolmogorov Entropy for Gaussian Measures
Let U denote the unit ball of the Cameron-Martin space of a Gaussian measure on a Hilbert space. The sharp asymptotics for the Kolmogorov (metric) entropy numbers of U is derived. The condition imposed is regular variation of the eigenvalues of the covariance operator. A consequence is a precise link including constants to the functional quantiza-tion problem.
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Entropy
سال: 2011
ISSN: 1099-4300
DOI: 10.3390/e13030595